Medical user interfaces and related methods of use

ABSTRACT

A medical system for use in a lithotripsy procedure may include a processor configured to receive input from a first imaging device, wherein the first imaging device may be configured to send image data representative of an image captured in a lumen of a kidney, bladder, or ureter to the processor. The processor may be configured to display the image on a display device coupled to the processor, and analyze the image to sense the presence of an object within the image. If an object was sensed within the image, the processor may analyze the image to estimate a size of the object, and display the estimate on the display device.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This patent application is a continuation of U.S. application Ser. No.16/925,694 filed on Jul. 10, 2020, which is a continuation of Ser. No.16/290,430 filed on Mar. 1, 2019, now U.S. Pat. No. 10,743,946, which isa continuation of U.S. application Ser. No. 15/416,838, filed Jan. 26,2017, now U.S. Pat. No. 10,258,415, which claims the benefit under 35U.S.C. § 119 to U.S. Provisional Patent Application No. 62/288,654,filed on Jan. 29, 2016, all of which are incorporated by referenceherein in their entireties.

TECHNICAL FIELD

Examples of the present disclosure relate generally to medical userinterfaces and related methods of use. More particularly, examples ofthe present disclosure relate to medical user interfaces for use inlithotripsy procedures.

BACKGROUND

Many patients develop stones within their common bile, urinary, renal orureteral systems. These stones may block ducts and cause great pain andtherefore must be removed. Several approaches are available for treatingsuch stones, including, laser treatment and subsequent removal of thestones from the body. These lithotripsy procedures are often performedwith the aid of an endoscopic device, and an endoscopic operating fieldmay be displayed to an operator. Sizing objects in the endoscopicoperating field may be important to the treatment. However, currentmethods of sizing objects in the operating field are inaccurate and/orcumbersome.

Thus, there remains a need for improved methods and devices for sizingobjects during lithotripsy procedures.

SUMMARY OF THE DISCLOSURE

In one aspect, the present disclosure is directed to a medical systemfor use in a lithotripsy procedure. The medical system may include aprocessor configured to receive input from a first imaging device,wherein the first imaging device may be configured to send image datarepresentative of an image captured in a lumen of a kidney, bladder, orureter to the processor. The processor may be configured to display theimage on a display device coupled to the processor, and analyze theimage to sense the presence of an object within the image. If an objectwas sensed within the image, the processor may analyze the image toestimate a size of the object, and display the estimate on the displaydevice.

The processor may be configured to receive a virtual boundary of theobject on the displayed image from a user, and wherein the processor maybe configured to estimate the size of the object based on the virtualboundary. The processor may be configured to compare the image to adatabase of images to determine the presence of the object within theimage. The object may be a ureteral orifice. The object may be a stone.The processor may be configured to update the estimate of the size ofthe stone in real-time while fragments of the stone are being removed bythe application of a laser to the stone. The processor may be configuredto update the estimate of the size of the stone based upon a proportionof the image that is occupied by the stone. The processor may beconfigured to analyze the image to sense when fragments of the stonehave been removed by the laser. The processor may be configured toanalyze the image to estimate a size of the removed fragments. Theprocessor may be configured to generate a scaled grid that is configuredto be overlaidoverlaid onto the image displayed on the display device.The scaled grid may be representative of an actual size of objectswithin the image. The processor may be configured to regenerate thescaled grid whenever the processor senses that the first imaging devicehas been moved. The processor may be configured to receive x-ray datarepresentative of an x-ray image from an x-ray imaging device, and maybe configured to simultaneously display the image alongside the x-rayimage on the display device. The object may be an orifice, and whereinthe processor may be further configured to analyze the image containingthe orifice and generate recommended parameters for inflating a balloonthat is inserted into the orifice. The recommended parameters mayinclude a length of time for inflating the balloon.

In another aspect, the present disclosure is directed to a medicalsystem for use in a lithotripsy procedure performed on a patient. Themedical system may include a processor configured to receive input froma first imaging device, wherein the first imaging device may beconfigured to send image data representative of an image captured in alumen of a kidney, bladder, or ureter to the processor. The processormay be configured to display, on a display device coupled to theprocessor, an ablation user interface during an ablation portion of thelithotripsy procedure, the ablation portion of the lithotripsy procedureincluding inserting a balloon into the patient, wherein the processormay be configured to simultaneously display an image of the balloondisposed within the patient and a recommendation for a length of time toinflate the balloon on the ablation user interface.

The medical system may further include a second imaging device that isan x-ray imaging device configured to send x-ray data representative ofan x-ray image to the processor. The processor may be further configuredto display, on the display device, a first user interface beforeinitiation of the lithotripsy procedure displaying information regardingthe patient, and display, on the display device, a second user interfaceduring a first portion of the lithotripsy procedure, wherein the seconduser interface may include one or more of the image and the x-ray image.The processor may be further configured to display, on the displaydevice, a third user interface during a second portion of thelithotripsy procedure, the second portion of the lithotripsy procedureincluding inserting a guidewire into the lumen, and wherein the thirduser interface may include both an image and an x-ray image of theguidewire displayed simultaneously on the display device. The processormay be further configured to display, on the display device, a fifthuser interface during a fourth portion of the lithotripsy procedure,wherein the fourth portion of the lithotripsy procedure may includeablating a stone with laser energy, wherein the fifth user interface mayinclude an image of an energy delivery element delivering laser energyto the stone within the lumen. The processor may be configured todisplay, on the display device, a sixth user interface during a fifthportion of the lithotripsy procedure, wherein the fifth portion of thelithotripsy procedure may include delivering a stent into a lumen of thepatient, wherein the sixth user interface may include an image of thestent while disposed within the patient. The processor may be furtherconfigured to display, on the display device, a seventh user interfaceafter the lithotripsy procedure has concluded, wherein the seventh userinterface may include data relating to the one or more of the first,second, third, ablation, or fifth portions of the lithotripsy procedure.

In yet another aspect, the present disclosure is directed to a methodfor displaying images relating to a lithotripsy procedure. The methodmay include analyzing an image received from an imaging device disposedwithin a lumen of a patient to sense the presence of an object withinthe image. If the object is sensed within the image, the method mayfurther include analyzing the image to estimate a size of the object,and displaying the estimate on the display device.

The object may be a stone, and the method may further include updatingthe estimate of the size of the stone in real-time while fragments ofthe stone are being removed by the application of a laser to the stone.The method may further include analyzing the image to sense whenfragments of the stone have been removed by the laser.

Additional objects and advantages of the disclosure will be set forth inpart in the description which follows, and in part will be obvious fromthe description, or may be learned by practice of the disclosure. Theobjects and advantages of the disclosure will be realized and attainedby means of the elements and combinations particularly pointed out inthe appended claims.

The accompanying drawings, which are incorporated in and constitute apart of this specification, illustrate several examples of the presentdisclosure and together with the description, serve to explain theprinciples of the invention.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic view of a system for performing a lithotripsyprocedure according to one example of the present disclosure.

FIGS. 2-11 are exemplary user interfaces that may be used with thesystem of FIG. 1 .

DESCRIPTION OF THE EXAMPLES

Reference will now be made in detail to examples of the presentdisclosure, which are illustrated in the accompanying drawings. Whereverpossible, the same reference numbers will be used throughout thedrawings to refer to the same or like parts.

Examples of the present disclosure relate to devices and methods forcontrolling the application of energy to objects disposed within a bodylumen of a patient, such as, e.g., a lumen of a kidney, a bladder, or aureter. FIG. 1 illustrates a system 100 for delivering energy, inaccordance with a first example of the present disclosure. The systemmay include a processor 102 that is operatively coupled to a display106. In some examples, processor 102 and display 106 may be disposedwithin a single handheld unit, such as, e.g., a tablet computer such asa Microsoft Surface, iPAD® or iPHONE®. In other examples, processor 102and display 106 may be modular and may connect to one another by anysuitable mechanism. Display 106 may be a touchscreen input device thatallows a user to send commands to processor 102. In other examples, amouse and/or keyboard (not shown) may be operatively coupled toprocessor 102. Multiple display devices (with or without inputcapability) may be deployed at alternate sites in or out of theoperating suite. This may include video output streams for broadcast toalternate pre-exiting/third party displays/locations.

Processor 102 also may be coupled to energy delivery electronics 108, anendoscopic device 110, a fluoroscope 112, an x-ray imaging device 114,an ultrasound device 116, a fluid delivery device 118, and a patientmonitoring device 120.

Processor 102 may be generally configured to accept information from thesystem and system components, and process the information according tovarious algorithms to produce control signals for controlling energydelivery electronics 108, endoscopic device 110, fluoroscope 112, x-rayimaging device 114, ultrasound device 116, fluid delivery device 118,and patient monitoring device 120. The processor 102 may acceptinformation from the system and system components, process theinformation according to various algorithms, and produce informationsignals that may be directed to visual indicators, digital displays,audio tone generators, or other indicators of, e.g., a user interface,in order to inform a user of the system status, component status,procedure status or any other useful information that is being monitoredby the system. The processor 102 may be a digital IC processor, analogprocessor or any other suitable logic or control system that carries outthe control algorithms.

Energy delivery electronics 108 may include an optical energy source,such as, e.g., a holmium (Ho) laser source, a holmium:YAG (Ho:YAG) lasersource, a neodymium-doped:YAG (Nd:YAG) laser source, a semiconductorlaser diode, a potassium-titanyl phosphate crystal (KTP) laser source, acarbon dioxide (CO2) laser source, an Argon laser source, an Excimerlaser source, a diode laser source, or another suitable laser source. Insome examples, the laser source may be a laser diode. The laser diodemay illuminate a target area, and may be mounted at the distal end of acatheter or other suitable elongate member, such as, e.g., endoscopicmember 110. In some examples, a high power (e.g., superluminescent) LEDmay be used in place of a laser source. In some examples, an intense,pulsed light source may be used in place of a laser source.

In an alternative example, energy delivery electronics 108 may be apneumatic control device for performing lithotripsy procedures by directcontact of a probe with a targeted stone. In this alternative example,processor 102 may control air pressure and frequency as well asirrigation activation and flowrate. The processor 102 may also be ableto connect via the network 140 to the hospital server 142 and obtainpatient related data from the HIS, PACS and EMR. This data can then beprocessed and displayed on the display device 106. The system may alsoallow the processor 102 to send updated data based on the procedurestatistics and information back to the HIS, PACS and EMR.

Energy delivery electronics 108 may be configured to control delivery ofany other suitable energy modality, such as, e.g., ultrasound.

Endoscopic device 110 may be any suitable endoscopic member, such as,e.g., an endoscope, a ureteroscope, a nephroscope, a colonoscope, ahysteroscope, a uteroscope, a bronchoscope, a cystoscope, a sheath, or acatheter. Endoscopic device 110 may include one or more additionallumens configured for the passage of a variety of surgical equipment,including, but not limited to, imaging devices and tools for irrigation,vacuum suctioning, biopsies, and drug delivery. At least a portion ofendoscopic device 110 may be radiopaque.

An energy delivery fiber 126 (e.g., a laser fiber) may extend fromdistal end 122 of endoscopic device 110 through a lumen 124, and may beconfigured to deliver laser energy 127 at a distal end. Energy deliveryfiber 126 may include a light source channel that is configured toreceive light or laser energy at a proximal end, and transmit the lightor laser energy to a distal end via internal reflection within thefiber. Energy delivery fiber 126 may receive energy transmitted fromenergy delivery electronics 108, and may deliver the received energy toa desired treatment location, such as, e.g., a stone.

An imaging device 128 may also be disposed at the distal end 122 ofendoscopic device 110. The imaging device may include any suitabledevice configured to provide images to processor 102 for display ondisplay 106 including, e.g., a CMOS imaging sensor or other solid statedevice and one or more glass or polymeric lenses that produce electronicimage signals representative of an image of the tissue or other objectsin front of the imaging device 128. The imaging device 128 may be a lowlight sensitive, low noise video VGA, CMOS, color imager or higherresolution sensor such as SVGA, SXGA, or XGA. The video output of theimaging device 128 may be in any conventional format including PAL, NTSCor high definition video format, and may be transmitted to processor 102by any wired or wireless mechanism. X-ray imaging device 114 may detectthe x-rays generated by fluoroscope 112 and may be able to create anx-ray image of the treatment areas and of endoscopic device 110. Imageinformation detected by x-ray imaging device 114 may be provided toprocessor 102.

Fluid delivery system 118 may include an irrigation pump (not shown)that may be configured to provide irrigation fluid to a body lumenand/or to evacuate the fluid from the body lumen via lumens or channelsof endoscopic device 110. More specifically, activation of the pump by auser may send saline or another suitable irrigation fluid throughendoscopic device 110. Ultrasound device 116 may be any suitable deviceconfigured to produce a real-time ultrasound image of body tissues andlumens. Monitoring device 120 may include sensors configured to measurethe blood pressure, pulse rate, temperature, and peripheral capillaryoxygen saturation (SpO2), among other patient vitals. Portions ofmonitoring device 120 may be disposed on the skin of the patient, withinthe patient, or may be positioned off of the patient.

Processor 102 may be coupled to one or more servers 142 via a network140, such as, the Internet. Servers 142 may provide various informationto processor 102 such as, e.g., electronic medical records of thepatient, among other information. The electronic medical records mayinclude standard medical and clinical data gathered by one or morehealth care providers for the patient, and may constitute acomprehensive medical history for the patient. The electronic medicalrecords may include patient information pertinent to the physicianperforming a lithotripsy procedure, such as, previous history of stones,allergies to anesthesia, whether the patient is on blood thinners, amongother information.

FIGS. 2-11 depict various user interfaces that may be shown on display106 over the course of a medical procedure, such as, e.g., during alithotripsy procedure for breaking and removing stones from the kidney,bladder, or ureter. The user interfaces may allow a physician to performthe various stages of a lithotripsy procedure using a single display andcontrol console.

FIG. 2 depicts an exemplary user interface displayed on display 106before a lithotripsy procedure is initiated. The user interface mayinclude a procedure section 150 having multiple tabs that may beselected by a user, via e.g., a touchscreen of display 106. The tabs maycorrespond to various user interfaces usable by a physician or operatorduring different portions of the lithotripsy procedure. For example, tab152 may correspond to a first portion of the lithotripsy procedure, suchas a cystoscopic evaluation portion of the lithotripsy procedure.Additional tabs 154, 156, 158, and 160 (tab 160 shown only in FIGS. 8-11) may also be shown which correspond to other portions of thelithotripsy procedure. For example, tab 154 may correspond to a guidewire insertion portion of the lithotripsy procedure, while tab 156 maycorrespond to a balloon dilation portion of the lithotripsy procedure.Further, tab 158 may correspond to a stone fragmentation portion of thelithotripsy procedure, while tab 160 may correspond to a stentingportion of the lithotripsy procedure. The user interface may alsoinclude a patient information tab 162 and a summary tab 164. One or moreof tabs 152-164 may be displayed concurrently to a user, and may beselected by the user at any time. The selection of a given tab by theuser may cause different user interfaces to be displayed on display 106.It is also contemplated that additional portions of the procedure may beembodied in additional tabs selectable by a user.

Referring specifically to FIG. 2 , patient information tab 162 is shownto be selected, and a corresponding patient information user interfaceis displayed on display 106. The patient information user interface mayinclude a procedure timer 166 and a patient vitals field 170. Thepatient information user interface may also include a patient statsfield 172, a diagnosis/anatomy field 174, a details field 176, and anx-ray field 178.

In some examples, procedure timer 166 may display an elapsed time of thelithotripsy procedure. Procedure timer 166 may be started manually by anoperator, or may start automatically upon a trigger event, such as, forexample, when a patient is administered anesthesia. Procedure timer 166may be displayed on all user interfaces during all portions of theprocedure. Patient vitals field 170 may display one or more patientvitals collected by, e.g., patient monitoring device 120, among otherdevices. For example, one or more of the patient's heart rate, bloodpressure (e.g., noninvasive blood pressure (NIBP)), and/or peripheralcapillary oxygen saturation (SpO2), among other patient vitals, may bedisplayed in patient vitals field 170.

Patient stats field 172 may display one or more statistics relating tothe patient undergoing the lithotripsy procedure which may be retrievedby processor 102 from servers 142 via network 140. One or more of thepatient's photo, name, age, gender, date of birth, primary physician,specialty physician, among other information, may be provided in patientstats field 172. Diagnosis/anatomy field 174 may include informationrelating to a previous diagnosis of the patient relating to thelithotripsy procedure. For example, a visual representation of thekidneys may be shown and the kidney diagnosed as having a stone, e.g.,the right kidney, may be highlighted in diagnosis/anatomy field 174.Additionally, other information relating to the diagnosed stones, suchas a pre-procedure estimate of the stone size may also be displayed indiagnosis/anatomy field 174. Details field 176 may include otherinformation pertinent to the lithotripsy procedure. For example, medicaldata specific to the patient that is relevant during a surgicalprocedure may displayed, such as, e.g., whether the patient is on bloodthinners, whether the patient is allergic to anesthesia, whether thepatient's ureter is known to be narrow, and whether the patient has aprior history of kidney stones. An x-ray field 178 may also be shown onthe patient information user interface. X-ray field 178 may allow anoperator to select from one or more previously taken x-ray images of thepatient so that an operator may view those images and familiarizeherself with the patient's anatomy prior to the start of the lithotripsyprocedure.

A user interface is shown in FIG. 3 which may correspond to anevaluation portion of the lithotripsy procedure. For example, theprocedure may start with a visual evaluation of the kidney, bladder, orureter, using an imaging device coupled to a cystoscope, ureteroscope,or the like. At this stage, the operator may select evaluation tab 152to display one or more of a camera field 184 and an x-ray field 188 asshown in FIG. 3 . In the example user interface shown in FIG. 3 , thecamera field 184 is shown as maximized, while only a portion of thex-ray field 188 is shown. This configuration may allow an operator toview a camera image 186 generated by, e.g., imaging device 128 shown inFIG. 1 , on a larger proportion of display 106 for additional clarity.The user may select that x-ray field 188 be simultaneously displayedwith camera field 184, or alternatively displayed instead of camerafield 184, if desired. X-ray field 188 may be generated in real time byfluoroscope 112 and x-ray imaging device 114. The camera field 184 mayallow a physician to visually perceive stone size and hardness, and mayalso allow the physician to determine the position of the energydelivery fiber 126 relative to the imaging device 128.

Once the lithotripsy procedure has begun, the user interface may furtherdisplay a grid icon 182 and a mapping icon 180 that will be discussed infurther detail below.

FIG. 4 depicts a user interface displayed on display 106 when anoperator activates grid icon 182. In response to grid icon 182 beingselected by a user during an evaluation portion of the lithotripsyprocedure, a scaled grid 190 may be overlaidoverlaid onto at least aportion (or all) of camera image 186. The scaled grid may have one ormore markings to allow an operator to quickly determine the actual sizeof an orifice, lumen, stone, or other object displayed on camera image186. For example, the grid 190 may include a plurality of boxes, and thelength of each box may correspond to an estimated distance within thelumen being displayed on camera image 186. For example, the length ofone of the boxes on camera image 186 may correspond to, e.g., 1 mmwithin the lumen. The scaled grid 190 may be developed by processor 102in real time using any suitable mechanism, including a reference objectof known size in the viewing field, among other techniques. The scaledgrid 190 may be updated in real time as the imaging device 128 is movedthrough the body in order to ensure accuracy. That is, each movement ofimaging device 128 may require that the scaled grid 190 be reconfigured.For example, while disposed in a first position, the length of each boxof scaled grid 190 may correspond to 1 mm within the lumen, but when theimaging device is moved to a second position (e.g., further distally inthe lumen), the length of each box of scaled grid 190 may correspond to1.2 mm within the lumen. In some examples, the length of each box onscaled grid 190 may be changed in order to represent the same distancewithin the image (e.g., 1 mm). In the example described above, thelength of the boxes of scaled grid 190 would be larger when the imagingdevice is disposed in the first position than when the imaging device isdisposed in the second position. In some examples, capture of an objectof fixed reference may be needed to calculate and project the grid. Toavoid depth based distortion, this may need to be calculated at a knowndistance or distances from the scope tip. Thus, the input of the devicesvisible in the surgical field (e.g., guidewire, basket, laser fiber) maybe used. These devices may have markings on their shafts that wouldidentify distance from scope tip that would be machine readable andindependent of scope being used. These algorithms could be based onpre-programmed values for a given device(s) or through real-timeacquisition from video stream. The scaled grid may be generated by thecomputing system run by processor 102 as a smart layer over the cameraimage 186. The unit size of each box of the scaled grid 190 may have afixed base reference value which may be for example 1 mm. This value maybe scaled in proportion to the camera image keeping the overall aspectratio fixed.

FIG. 5 depicts a user interface displayed on display 106 when anoperator activates mapping icon 180. In response to mapping icon 180being selected by a user, processor 102 may be configured to determine asize of objects disposed within camera field 186. In the image depictedin FIG. 5 , two orifices or openings are shown in camera field 186, afirst orifice 194 and a second orifice 196. The processor 102 may beconfigured to recognize the presence of a certain object in the viewingfield, such as, e.g., first and second orifices 194 and 196 based onvarious differentiating features of those orifices. For example,processor 102 may be configured to recognize the color, shape, or otherfeature of the orifices. Processor 102 may also be configured to comparecaptured images with a database of images to make determinations basedon the comparisons. For example, processor 102 may be able to identifyorifices, stones, or other objects in the viewing field based on thesimilarity of those objects with images stored in a database accessibleby process 102. In some examples, however, the user may be required todelineate the outer boundary of the object to be sized, for example, bydrawing or otherwise placing virtual boundaries around the objects to besized. For example, the user may place a first boundary 198 around theimage of first lumen 194, and may place a second boundary 200 around theimage of second lumen 196. Once the boundaries are placed around theobject to be sized, processor 102 may estimate or otherwise determinethe size (e.g., diameter) of first orifice 194 and second orifice 196.The determined sizes of those orifices may be overlaidoverlaid ontocamera image 186 as values 202 and 204, or may be displayed in anothersuitable location. For example, the values 202 and 204 may be displayedin a chart or other graphic on display 106 but outside of camera image186. In one example, process 102 may utilize a reference object of aknown size in the field of view to determine the size of another objectin the field of view.

FIG. 6 depicts an exemplary user interface that may be shown on display106 during a second portion of the lithotripsy procedure, for example,during guide wire insertion. During this stage of the procedure, anoperator may position a guidewire through portions of the ureter,bladder, and/or kidney to serve as a track for inserting other tools.The operator may activate tab 154 before inserting the guidewire intothe patient, and the activation of tab 154 may cause camera field 184 todisplay concurrently with x-ray field 188. The operator may then be ableto visualize an image of a guidewire 208 collected by, e.g., imagingdevice 128, while concurrently viewing a representation of guidewire 208on x-ray field 188. Processor 102 may also be configured to detect thedistal end of the guidewire 208 on x-ray field 188, and may place anindicator 210 (e.g., an arrow) at the distal end of the guidewire.Processor 102 may update the position of indicator 210 in real time asthe distal end of the guidewire moves through the body.

In some lithotripsy procedures, various access points may be too smallfor the necessary equipment to reach the intended treatment locations,such as, the locations of stones. In such examples, the access points(e.g., orifices and/or lumens) may be enlarged by balloon dilation. FIG.7 depicts a user interface shown on display 106 during a balloondilation portion of a lithotripsy procedure. The user interface of FIG.7 may be shown on display 106 after an operator activates tab 156. Insuch cases, camera field 184 may be enlarged such that a balloon 229disposed within the field of view of imaging device 128 is shown ondisplay 106.

During this portion of the procedure, an operator may select mappingicon 180, which may cause processor 102 to analyze the image 186 torecommend a dilation size and length of time that the balloon 229 shouldbe maintained in the dilation mode. For example, a first representation230 may be overlaidoverlaid on camera image 186, and may represent anestimated or measured size of a body orifice or lumen in the field ofview. A second representation 232 may also be overlaidoverlaid on cameraimage 186 to show the user the size of the lumen that is needed toinsert necessary instrumentation through the orifice or lumen. Processor102 may receive the desired lumen size prior to the start of thelithotripsy procedure or during the lithotripsy procedure from theoperator. Processor 102 may also be configured to display arecommendation 234 on the user interface which may correspond to arecommended time for the operator to inflate balloon 229.

The calculation for the suggested time required to dilate the lumenusing balloon 229, may require the processor to calculate the differencebetween the first representation 230 and second representation 232. Theinbuilt algorithm may use this data to compare with a standard set ofvalues defined for balloon dilation of the lumen. For example, if thedifference is 4 Fr (14 Fr minus 10 Fr) as seen in the camera image 186(e.g., FIG. 7 ), the estimated dilation time as suggested by thealgorithm may be about 3 minutes. This would be enabled by comparing thevalue of 4 Fr with the standard value of dilation already built into thesystem information. In some examples, a maximum delta between actual anddesired ID may also trigger a safety alert and advice not to proceed.

While processor 102 may provide recommendations to the operator, theoperator may be required to enter a separate instruction to dilateballoon 229 to a certain diameter and for a certain amount of time. Inother examples, processor 102 may be configured to proceed to inflateballoon 229 automatically according to this recommendation.

FIG. 8 depicts an exemplary user interface that may be displayed duringa stone fragmentation portion of the lithotripsy procedure. This userinterface may be displayed once an operator has guided endoscopic device110 over guidewire 208 to the desired treatment site, and has alsoactivated or selected stone fragmentation tab 158 from procedure section150 of the user interface. The stone fragmentation user interface mayinclude camera field 184, a fragmentation stats field 300, and a mappingfield 302.

At this stage, camera field 184 may depict energy delivery fiber 126,laser energy 127 (during laser delivery), and a stone 318. Fragmentationstats field 300 may depict various parameters and statistics useful tothe operator during laser delivery. For example, the average brokenstone size 303, a stone removal percentage 304, an amount of saline used306, the total amount of energy delivered 308, the rate of energydelivery 310, the status of energy delivery 312, the current power levelof the laser 314, and an indicator 316 may be shown in fragmentationstats field 300. It is also contemplated that other pertinentinformation may also be displayed in fragmentation stats field 300.Stone density and type can be input ahead of the procedure based on adiagnostic workup. Stone density may be acquired via analysis of x-raydata preoperatively. Size is also acquired via preoperative CT,preoperative fluoroscopy and/or direct vision. From this information apreloaded table of settings can provide initial settings. As lithotripsyproceeds, the rate and size distribution of fragment generation can becalculated in real time, which may then generate new settings thatmaximize efficiency of removal. These may be implemented as pop upprompts or as auto-escalating settings.

The processor 102 may be connected to energy device electronics 108 viaa CAN BUS connector which may directly fetch the data regarding theenergy delivery parameters, and display the same on the display device106. The CAN BUS connection may also remotely control of the energydelivery parameters on the energy device electronics 108 enabling theurologist direct easy control of the laser energy settings

The average broken stone size field 303 may be calculated by processor102 by analyzing the stone fragments that detach from stone 318. Thatis, processor 102 may run image analysis software that determines thesize of stone fragments that detach from stone 318, and may display theaverage size of those fragments in field 303. The size of each stonefragment may also be displayed on fragmentation stats field 300, or maybe saved by processor 102 for further analysis. The stone removalpercentage field 304 may be updated by processor 102 in a similarmanner. For example, prior to the initiation of laser delivery tofragment stone 318, processor 102 may run image analysis software of oneor more collected images of stone 318 to estimate its size. After laserdelivery to stone 318 commences, this value may be continuously updatedby continued image analysis as fragments of stone 318 are removed. Thatis, processor 102 may determine that the stone size is decreasing basedon the reduced proportion of the camera field being occupied by thestone.

The fragments of the stone 318 which are flushed out of the system usingthe used saline fluid may be collected in a strainer. The weight ofthese fragments may also be measured and communicated to the processor102. The processor 102 may then compare the weight of the fragmentscollected with the estimated weight of the stone obtained from initialpatient diagnosis or X ray reports as obtained from the HIS and/or EMR.The information regarding the percentage of stone fragments may bedisplayed on the display device 106 at stone removal percentage field304.

Processor 102 may update the amount of saline used field 306 based on aninput from fluid delivery device 118. Processor 102 may update the totalamount of energy delivered 308, the rate of energy delivery 310, thestatus of energy delivery 312, the current power level of the laser 314,and an indicator 316 based on input from energy delivery electronics108. Indicator 316 may indicate to the user using one or more colors,whether energy delivery electronics 108 may be activated to deliverenergy. For example, a first color, e.g., green, may indicate thatenergy delivery electronics 108 may be ready to activate while a secondcolor, e.g., red, may indicate that energy delivery electronics is notready to be activated.

Mapping field 302 may include one or more virtual images of the kidneys,bladder, and/or ureter. In the example shown in FIG. 8 , a virtual image320 of a kidney is shown along with an enlarged virtual image 322 of aportion of the virtual image 320. The virtual images 320 and 322 mayshow a representation of a first kidney stone 318, along with a virtualimage of a second kidney stone 324. The virtual images 320 and 322 mayrepresent a virtual map of the kidneys, bladder, and/or ureter. As theendoscopic device 110 advances through the patient, a locating implementon the endoscopic device 110 may communicate with processor 102 toprovide a real position of the device. So long as the virtual mapcorrelates with the actual structure of the organ, an accurate virtualposition of the endoscopic device 110 can be displayed on the virtualmap on virtual images 320 and 322.

In some examples, treatment parameters of the lithotripsy procedure maybe associated with their respective treatment sites to generate atreatment history profile for the patient. This information may berecorded in a memory coupled to processor 102, and may be furthertransmitted to one or more servers 142 over network 140. This previoustreatment history profile may be loaded for use during a subsequenttreatment session. Still images or video clips at any point may also becaptured locally, removed locally (via, e.g., a thumb-drive), or pushedto cloud or EMR storage.

The parameters may include any of those parameters of values describedabove, including, but not limited to time or duration of treatment,temperature (of the site, the device, or an adjacent site), energy,power, average power, status/incomplete activations, position oftreatment, total energy applied, rate of change in temperature, rate ofchange in applied energy, amount of saline used, size of stone, averagesize of broken stone fragments, or a combination thereof. As notedherein, such parameters may be associated with the treatment locationswhere the parameters were recorded.

As shown in FIG. 8 , the virtual images 320 and 322 may track theprogress of the endoscopic device 110 as it advances through the kidney.The virtual images 320 and 322 may also visually differentiate treatedareas (for example, by shading treated areas) from untreated areas. Inaddition, the virtual images 320 and 322 may provide visual informationto guide the practitioner to the next site (e.g., via a virtual track326).

After stone fragmentation, an operator may choose to place a stent inthe ureter of the patient. In such examples, the operator may selectstenting tab 160 to have the user interface shown in FIG. 9 displayed ondisplay 106. This user interface may be substantially similar topreviously-described user interfaces, such as the user interfacedescribed with reference to FIG. 6 . The user interface of FIG. 9 usedduring a stenting portion of a lithotripsy procedure may simultaneouslydisplay a camera field 184 and x-ray field 188. This combination offields may allow a practitioner to visualize an image of a stent 327 tobe placed in the ureter, along with a real-time x-ray image of the stentin x-ray field 188.

After the conclusion of the lithotripsy procedure, an operator mayselect summary tab 164, which may cause display 106 to show the userinterface depicted in FIG. 10 . The summary user interface may include aprocedural stats field 340, a further information field 342, and anx-ray field 344. The procedural stats field 340 may display variousinformation such as, e.g., whether a stent was placed, the stone freerate (SFR), the possibility of recurrence of a kidney stone, the patientvitals recorded during the lithotripsy procedure, and whether thepatient was kept post-operatively at the treatment facility, among otherinformation. The further information field 340 may display other notesinput by the operator over the course of the procedure or after theprocedure, such as, e.g., whether an infection was observed within thebladder, kidney, or ureter, whether antibiotics were prescribed, and/orwhen an implanted stent should be removed. The x-ray field 344 may allowan operator to select one or more x-ray images that were taken duringthe course of the lithotripsy procedure. In some examples, the operatormay view the x-rays taken during the lithotripsy procedure as a video ofconsecutive images.

FIG. 11 depicts another user interface that may be accessed after theconclusion of a lithotripsy procedure. The user interface may include asummary field 346 which may display, among other information, theduration of the procedure, the type of stone removed (e.g., a calciumoxalate stone), the percentage of the stone removed, the size of thestone removed, and the various energy settings used during theprocedure.

The system and user interfaces disclosed herein may enable physiciansand technicians to make faster and smarter decisions to enhance clinicaloutcomes, reduce procedure time, and reduce cognitive loads requiredduring procedures.

Other examples of the present disclosure will be apparent to thoseskilled in the art from consideration of the specification and practiceof the present disclosure disclosed herein. It is intended that thespecification and examples be considered as exemplary only, with a truescope and spirit of the present disclosure being indicated by thefollowing claims.

What is claimed is:
 1. A medical system for use in a medical procedure,comprising: a first imaging device; a display device; and a processorconfigured to receive input from the first imaging device, wherein thefirst imaging device is configured to send image data representative ofa first image captured within a body lumen of a patient to theprocessor, wherein the processor is coupled to the display device,wherein the processor is configured to: analyze the first image to sensea presence of an object within the first image; when an object is sensedwithin the first image, analyze the first image to estimate a size ofthe object; display, on the display device, a first user interfaceduring a first portion of the medical procedure, wherein the first userinterface includes the first image and the estimated size of the object;display, on the display device, a second user interface during a secondportion of the medical procedure, the second portion of the medicalprocedure including inserting a guidewire into the lumen, and whereinthe second user interface includes both a second image and a first x-rayimage of the guidewire displayed simultaneously on the display device;display, on the display device, a third user interface during a thirdportion of the medical procedure, the third portion of the medicalprocedure including estimating a size of an orifice within a third imageor a second x-ray image, and wherein the third user interface includesthe third image or the second x-ray image and the estimated size of theorifice overlaid as a numerical value on either the third image or thesecond x-ray image; and display, on the display device, a fourth userinterface during a fourth portion of the medical procedure, wherein thefourth portion of the medical procedure includes application of laserenergy, and wherein the fourth user interface includes an image of anenergy delivery element delivering laser energy to the object within thelumen, wherein the processor is configured to analyze the image of theenergy delivery element delivering laser energy to the object within thelumen to sense whether fragments of the object have been removed by thelaser energy, and wherein the processor is configured to determine thata size of the object is decreasing based on a reduced proportion of acamera field being occupied by the object.
 2. The medical system ofclaim 1, wherein, if the processor senses that fragments of the objecthave been removed by the laser energy, the processor is configured toupdate the estimated size of the object.
 3. The medical system of claim1, further comprising: displaying an initial user interface beforeinitiation of the medical procedure, wherein the initial user interfacedisplays information regarding the patient, including at least one of apatient vitals field, a patient stats field, and a diagnosis or anatomyfield.
 4. The medical system of claim 1, wherein the estimated size ofthe object is overlaid as a numerical value on the first user interface.5. The medical system of claim 1, wherein the image data includes animage of a lumen of a kidney, a bladder, or a ureter.
 6. The medicalsystem of claim 1, wherein the processor is further configured todisplay, on the display device, another user interface during anotherportion of the medical procedure, wherein the another portion of themedical procedure includes delivering a stent into a lumen of thepatient, and wherein the another user interface includes an image of thestent while delivered within the lumen of the patient.
 7. The medicalsystem of claim 1, wherein the first user interface is not displayedduring the second portion, the third portion, or the fourth portion ofthe medical procedure, wherein the second user interface is notdisplayed during the first portion, the third portion, or the fourthportion of the medical procedure, wherein the third user interface isnot displayed during the first portion, the second portion, or thefourth portion of the medical procedure, and wherein the fourth userinterface is not displayed during the first portion, the second portion,or the third portion of the medical procedure.
 8. The medical system ofclaim 1, wherein the processor is further configured to display, on thedisplay device, an end user interface after the medical procedure hasconcluded, wherein the end user interface includes data relating to oneor more of the first, second, or third portions of the medicalprocedure.
 9. The medical system of claim 1, wherein the processor isfurther configured to generate a scaled grid that is configured to beoverlaid onto one or more of the first, second, or third userinterfaces, wherein the scaled grid is representative of an actual sizeof the object within the first image.
 10. A medical system for use in amedical procedure, comprising: a first imaging device; a display device;and a processor configured to receive input from the first imagingdevice, wherein the first imaging device is configured to send imagedata representative of a first image captured within a body lumen of apatient to the processor, wherein the processor is coupled to thedisplay device, wherein the processor is configured to: analyze thefirst image to sense a presence of an object within the first image;when an object is sensed within the first image, analyze the first imageto estimate a size of the object; display, on the display device, afirst user interface during a first portion of the medical procedure,wherein the first user interface includes the first image and theestimated size of the object; display, on the display device, a seconduser interface during a second portion of the medical procedure, thesecond portion of the medical procedure including estimating a size ofan orifice within a second image or an x-ray image, and wherein thesecond user interface includes the second image or the x-ray image andthe estimated size of the orifice overlaid as a numerical value oneither the second image or the x-ray image; and display, on the displaydevice, a third user interface during a third portion of the medicalprocedure, wherein the third portion of the medical procedure includesapplication of laser energy, and wherein the third user interfaceincludes an image of an energy delivery element delivering laser energyto the object within the lumen, wherein the processor is configured toanalyze the image of the energy delivery element delivering laser energyto the object within the lumen to sense whether fragments of the objecthave been removed by the laser energy and to estimate a size of one ormore fragments, wherein the processor is configured to determine that asize of the object is decreasing based on a reduced proportion of acamera field being occupied by the object.
 11. The system of claim 10,wherein the object is a stone, and wherein the third portion of themedical procedure is a lithotripsy procedure.
 12. The system of claim10, wherein the processor is configured to generate a scaled grid thatis configured to be overlaid onto one or more of the first, second, orthird user interfaces, wherein the scaled grid is representative of anactual size of the object within the image.
 13. The system of claim 10,wherein the processor is further configured to display, on the displaydevice, another user interface during another portion of the medicalprocedure, wherein the another portion of the medical procedure includesdelivering a stent into a lumen of the patient.
 14. The system of claim13, wherein the another user interface includes an image of the stentwhile delivered within the lumen of the patient.
 15. The system of claim10, wherein the processor is further configured to display, on thedisplay device, an end user interface after the medical procedure hasconcluded, wherein the end user interface includes data relating to oneor more of the first, second, or third portions of the medicalprocedure.
 16. A medical system for use in a medical procedure,comprising: an imaging device; a display device; and a processorconfigured to receive input from the imaging device, wherein the imagingdevice is configured to send image data representative of an imagecaptured in a lumen to the processor, wherein the processor is coupledto the display device, wherein the processor is configured to: displaythe image on the display device; analyze the image to sense a presenceof one or more objects within the image; when one or more objects aresensed within the image, analyze the image to estimate a size of the oneor more objects; generate a scaled grid that is configured to beoverlaid onto the image displayed on the display device, wherein thescaled grid is representative of an actual size of the one or moreobjects within the image; update the estimate of the size of the one ormore objects in real-time, wherein the processor is configured to updatethe estimate of the size of the one or more objects based on aproportion of each image that is occupied by the one or more objects;and during application of laser energy, analyze one or more additionalimages to sense whether fragments of the one or more objects have beenremoved by the laser energy.
 17. The medical system of claim 16, whereinduring application of laser energy, the processor is configured todisplay an image of an energy delivery element delivering laser energyto the one or more objects.
 18. The medical system of claim 16, whereinthe processor is further configured to display the estimate of the sizeof the one or more objects on the display device as a numerical valueoverlaid onto the image.
 19. The medical system of claim 16, wherein theone or more objects are one or more stones, and wherein the applicationof laser energy is a lithotripsy procedure.
 20. The medical system ofclaim 16, wherein the processor is configured to regenerate the scaledgrid whenever the processor senses that the imaging device has beenmoved.